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Noting the increasing interest in qualitative 
research in general and ethnographic research in particular, the 
authors of this report clarify the place of the latter within the 
former. The authors presen six criteria for an ethnographic approach 
to education, illustrated with examples, and discuss ethnography's 
advantages and disadvantages. The six criteria presented are that (1) 
the problem be discovered in the field, (2) the senior investigator 
conduct the bulk of the field work, (3) the field work take at least 
a school year, (4) the senior investigator shall have had 
cross-cultural experience, (5) multiple research methods be utilized, 
and (6) the finished product present a wealth of primary data from 
the studied group. The authors next demonstrate the variety in 
qualitative research by describing seme dimensions on which 
qualitative studies vary. In the arsa of data collection, qualitative 
strategies can vary in at least five ways: (1) the number of sites 
studied, (2) the extent of researcher immersion in the situation 
studied, (3) the extent to which activities are structured in 
advance, (4) the nature of the observation frameworks, and (5) 
reporting strategies. Data analysis can follow one or more of three 
approaches: the intuitive, the procedural, or the intersubjective . 
(Author/PGD) 
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TO ETHNOGRAPH OR NOT TO ETHNOGRAPH? 
VARIETIES OF QUALITATIVE RESFARCH IN EDUCATION 

Introduction 

Recently, a federal project officer told us about a problem he had 
with a grants competition, A number of the proposals received were to 
conduct "ethnographic" studies • An anthropologist on the review committee 
systematically rejected such proposals because the methodologies described 
were not really ethnographic. The project officer, whose background was 
In quantitative methods, was confused* He v.'ondered what it took to make 
a proposal truly ethnographic and whether it really mattered • 

This story illustrates two trends in the use of qualitative methods 
in educational research and evaluation today. On the one hand, the^e is 
a growing acceptance of such methodologies, as evidenced by the attention 
given them in numerous articles, books, and AERA sessions. On the other 
hand, there is still a great deal of confusion about what these methodolo- 
gies are. Researchers, policy makers, and practitioners have been intro- 
duced to such phrases as ethnography, field methods, case studies, quali- 
tative methods, and naturalistic inquiry without getting a clear idea of 
which is which. 

One of the major sources of confusion concerns the nature of ethrio- 
p.raphy. The tradition of ethnographic research dates bark co World War I 
(Wax, 1971), There is a substantial interest in employin.G; this approarh 
for policy and evaluation purposes (Wolcotr, 1975; Knapp, 1979), We will 
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argue that ethnography represents a specialized approach to qualitative 
inquiry. Because there is a developed research community in the discipline 
of anthropology to learn from, this approach has received a great deal of 
attention. However, many of the people interested in ethnography do not 
understand the specialized meaning that anthropologists give to this word. 
There is, in fact, a greater variety of approaches to data collection and 
analysis in the field of qualitative inquiry than in ethnography, and it 
would be a mistake to view the two as synonymous. Moreover, ethnographv 
Ls more feasible and useful for conducting basic research Chan for policy 
and evaluation purposes. For more applied purposes, otner forms of quali- 
tative inquiry are more appropriate. 

This paper will place ethnography in the larger field of qualitative 
methods in educational research. After a brief description of qualitative 
methods, it presents a set of ^.riteria for an ethnographic approach to 
education along with examples and a discussion of the advantages and dis- 
advantages of ethnography. Next, it describes some dimensions on which 
qualitative studies vary to show the v>riety in the larger field. This 
overview should be useful for those designing evaluation and policy 
studies in education. 

Qualitative and Quantitative Inquiry 
Despite the fact that many people have written about the advantages, 
probhms, and experiences of using qualitative approaches, fev^; have de- 
lined 'V|ual i tative.*' Van Maanen (1979: 520) Sviys it ha> no precise mean- 
in,': ii*' -rs "it is .It best an umbrella tenti ♦overinr. an .nra.v <m interpret i 
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techniques," Nevertheless, the field of qualitative inquiry can be des- 
cribed in general terms. Qualitative inquiry is conducted primarily 
through observation and interviewinp,. Data collection plans, including 
the foci of a study and the specific methods of collecting and analyzing 
data, are flexible; they are not as highly structured in advance as in 
quantitative inquiry. Instead of relying on printed instruments which 
specify not only research questions but response alternatives, qualitative 
researchers "use themselves as the principal and most reliable instrument 
of observation, selection, coordination, and interpretation" (Sanday, 1979: 
528). In effect, researchers take advantage of their own subjective 
understandings of research settings as they gather and interpret informa- 
tion. The data of qualitative inquiry — that is, the data which are col- 
lected, interpreted, and communicated to others — ^are in the form of words 
rather than numbers. 

Qualitative and quantitative inquiry are often viewed as coming out 
of different paradigms which are incompatible, perhaps irreconcilably so. 
However, several researchers have suggested that the two approaches can be 
used together. Rist (1977) is pessimistic about combining the two but 
says that each has a place in the collective social science enterprise. 
They should be allowed to coexist peacefully; researchers should use both 
approaches to illuminate different portions of reality. Reichardt and 
Cook (1979) show that attributes associated with the qualitative paradigm 
can be found in quantitative research and vice versa. Dentin (1970) 
argues that every investigation should use multiple methods of inqtiiry, 
f^urh ns survey and participant observation, because each method has biases 
which can be counteracted by other methods. Sieber (197'5) points out that 
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qualitative and quantitative methods can be used to support each other and 
gives many examples of how this may occur. 

The Place of Ethnography 
This section presents criteria for identifying ethnographic studies 
of education and some examples of ethnographic studies published in the 
'70s. Then it discusses some advantages and disadvantages of ethnographic 
res earch. 

Criteria for ethnography 

In introducing a special issue on ethnography in education in th-^ 
anthropology journal, Human Organization , Harry Wolcott (1975) both defines 
ethnography and presents some criteria for knowing when a study is ethno- 
graphic. 

An ethnography is, literally, an anthropologist's "picture" 
of the way of life of some interacting human group. . . A 
deceptively simple test for judging the adequacy of an ethno- 
graphic account is to ask whether a person reading it could 
subsequently . . . anticipate and interpret what occurs in the 
group as appropriately as its own members (Wolcott, 1975:112). 

The idea that an ethnography should help the reader understand the rules 

by which members of the group govern and interpret their behavior is coming 

to be an accepted goal for ethnography (Sanday, ?979). Achieving this 

goal generally requires a fairly extensive product, rich in descriptive 

detail. Hence, the prototypical ethnographic report is a book-length case 

study of a single group. 

WoJcutt goes on to discuss several criteria for an ethnographic 

approach. \lv have derived six criteria from his discussion. First, the 
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proDlem is discovered in the field * This criterion is not meant to 
encourage conceptual nihilism or preclude planning. Certainly, Margaret 
Mead knew she was going to study child development before she ever left 
for the South Pacific (Mead, 1972), and Wolcott (1973) himself exhibited 
a great deal of advanced planning in his ethnography of a principal, if 
only by going through considerable pains in his selection of an appropri- 
ate subject. More generally, Goetz and LeCompte (1981) discuss the role 
of theory in the design of ethnographic research, including problem formu- 
lation. The point of this criterion, however, is that the ethnographer 
avoids rigidly prespecified hypotheses. Instead, the ethnographer prefers 
to "muddle around" in a setting, develop an understanding of the situa- 
tion, and use that understanding tc refine the initial research problem 
and formulate an interpretation. In the extreme case, the field situation 
may suggest an alternative problem, not aanicxpated when field work began, 
that is more fruitful to pursue. 

Second, the senior Investigator should conduct the bulk of the field 
work personally . Wolcott emphasizes the importance of having the person 
v^ho does the final write-up conduct the field work (see also Wax, 1971). 
Only through direct experience does the researcher develop the kind of 
personal understanding of the subjects that makes an ethnographic account 
possible*. Research assistants may be of use, but they are no substitute 
for direct participation in the research setting. Exper-^'ence mediated 
t-hrouRh a research assistant is no help in internalizing and understand- 
inw a si tiiat ion. 
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Third, field work should take at least a school year > Here again, 
the goal is to enhance the utility of the researcher as an instrument. 
Intensive immecsion, it is suggested, is really necessary to understand 
the rules, customs and conditions governing life in the group studied. 
A three-day site visit, a growing phenomenon among qualitative researchers, 
is simply not believed to be enough to break through the researcher's 
personal beliefs and norms and allow a direct experiencing of those of 
the group. Incidentally, Wolcott also suggests that the time required to 
write an ethnographic report is often seriously underestimated. As much 
time may be required for writeup as for the field work (Wax, 1971). 

tourth, the senior investigator should have had (or be having) a 
cross-cultural experience . The argument for this criterion is that the 
contrast provided by knowing another culture helps make manifest what 
members of the group studied take for granted. Wolcott quotes anthropolo- 
gist Clyde Kluckhohn (1949:16) who points out that "It would hardly be 
fish who discovered the existence of water." 

^^ifth, the ethnographic approach incorporates the use of multiple 
research methods . Two methods are especially central to ethnography: 
direct observation of the activity of the group and intensive interviewing 
of informants who can explain and provide an understanding of what takes 
place. However, these methods are rarely used alone. Ethnographers may 
also use structured interviews, surveys, life histories, collections of 
e-A'^tin^; documents and artifacts, unobtrusive measures, psycholo)[;ical 
trrhniques, videotape, photography and a variety of other means to put 
to).;ether a picture of the group studies. 
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Sixth, the finished ethnographic product presents a wealth of primary 
data from the group in quest ion ♦ Primary data are more than verbal des- 
cription. They include "works" produced by the subjects themselves in- 
cluding quotes from interviews, excerpts from documents, stories, songs, 
maps, and sketches. Products minimally structured by the observer, such 
as photographs, might also be included. 

Current ethnographic work 

There is in fact a growing body of ethnographic research on education 
in America. To give a flavor of the kind of work that is available, 
Table 1 presents information showing how well seven case studies published 
after 1970 meet the criteria for ethnography mentioned above. The list 
of case studies is illustrative, not comprehensive; and not all are in- 
tended to be ethnographies. Most of the studies are among the better 
known works in the field. Some are newer and their place is not yet es- 
tablished. In putting together the table, two observations beCcHne clear 
to us. First, there are a number of case studies available that meet 
most of the listed criteria. The criterion least frequently met is for 
cross-cultural experience. To our knowledge four of the sets of authors 
listed had no such experience as part of cheir prof p«?sional careers. On 
the other hand, one — John Ogbu — reverses the typical pattern. After 
decades of Europeans and Americans studying colonial and "primitive" 
cultures, he is a Nigerian who studies America. Problem identification 
typically began before field work started but was substantially affected 
by th^ field work. In four case studies, the authors knew the issues they 
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TITLE 



Problem Idencl- 
ficacion in Field 



Apnllrntinu of frltcrhi 
For Etlmoi;rjphy 

CRTTEJRION 

Senior Invest i- Time in 

r.acor's »^ield 
Field Acclvicv 



Anatomy of Educational Innovation: 
All Orjjanizational Analysis of an 
Elementary School (Smith Kc»ith, 
197n 



Proposal before 
started; problem 
reformulated 
over time 



Senior investi- 
gator with 
assistants 



1 school 
vear 



Great Expectations for Small Schools: After arrlvinj^ Senior Invest 1- 

The Limitations of Federal Projects In field gator did 907 

(Firestone, 1980) of work 



^ years 



Growlnc Up Air.prican: Schooling: and 
the Stirvival of Community 
(Peshkln, 1976) 



Problem broadly 
Identified before 
i^oing into field; 
refined during 
•5tudy 



Senior investi- 
gator with sev- 
eral assistants 



2 years 



Inside High School: The Student's 
Uorld (Cualck, 1973) 



Broad questions 
tr.imed before 
study began 



Senior investi- 
gator working 
alone 



1 year 



The Man in the Principal's 
Office: An Ethnography 
(Wolcott. 1973) 



Broad problem 
Identified before 
going into field 



Senior Investi- 
gator working 
alone 



1 year 



The Kext Ceneratlon: An 
Ethnography of Education 
in an Hrban Neighborhood 
(Ogbu. 1974) 



Problem identified 
after arriving in 
field 



Senior investi- 
gator working 
alopp 



2 vuars 



Organizing an Anarchy: 
Belief, Bureaucracy and 
Politics in Che National 
Institute cf Education 
(Sproull, ^Telner, U*olf. 1978) 



After partli'lpa- 
tion; before and 
during formal 
data collection 



3 Co-authors 
did most of 
work 



Worked In 
and around 
arencv. 
2 year-^ of 
nUorvlewing 



Invest iga- 

lor 's 
rro;>s- 
Tul tural 
Fxpcrience 



Methods 
Used 



health of 

Priraarv 

^Cxscription 



None 



Observation 

Interviewing 

Documents 



Quotes from field notes and 
docuinents. 



None 



Observation 

Interviewing 

Survey 

Documents 



Quotes from interview^:, ffeld 
notes, documents, and open- 
unded survuv Items, 
"specimen forms 



Nigeria, 
Pakistan 



Observation 
Interviewing 
Surveys 
Documents 



Ouotes from interviews, docu- 
ments, and field notes. 



None 



Observation 

Interviewing 

Documents 



Detailed description 
Ouotes from IntervU'w^i md 
observat Ions 



Torth 

American 

Indians 



Observation 
Interviev?ing 
Documents 
"Time ^ ^fotion•' 

study 
Ouestlonnaires 



Quotes from interviews, fluid 
notes, open-ended survev 
items, ind documents. 



An i,crlcan 
studying 
American 
education 



Interviewing 
Dv»cumcnts 
Obsurvat Ion 
Existing data 



Quotes from interviews and 
documents 



%'one 



Partlcipa- 

t ion 
Interviewing* 
Surveys 
Documents 



Qtio »s frf^m interviews, 
c'>ngress lona I t ust Inony , 
and dotMiments. 
Or^Mnl^ac li^n < i»irts 



intended to address before beginning the study, but refined the problem 
as they went — the typical app^roach. Sproull, Weiner, and Wolf became in- 
volved in a situation as participants before realizing they had a setting 
and issue worthy of study. Firestone's setting was selected by his employ- 
ing organization, and he was given no preparation time to formulate a 
problem. However, his employer also refused to specify a problem, thus 
giving him the opportunity to do so himself. The other criteria seem to 
be met in a fairly standard manner. 

The second observation concerns the wealth of subject matter and 
issues addressed in these studies. The studies represented here cover 
the full range of educational phenomena from the life of students (Cusick) 
to the machinations of the federal government (Sproull, et al.). There 
are studies of socialization (Cusick and Peshkin) , stratification (Ogbu) , 
and organizational processes (Wolcott, Smith and Keith); studies that look 
at how schools maintain the stability of the social order (Ogbu, Peshkin) 
and how they are changed by external forces (Firestone, Smith and Keith); 
studies that focus on a single individual (Wolcott) and whole district 
(Firestone) or agency (Sproull, et al.). Together they provide a rich 
body of knowledge on many aspects of the complicated Aiperican educational 
system. 

Advantag;es and d is advantages of ethnography 

Ethnographic research is especially useful for some purposes, for 
instance, basic research. Yet, it has serious limitations which make it 
a cuipbersome tool to use in its pure form for evaluative or policy 



research purposes. The advantages have to do with meaning, process, 
context, and evidence. 

Ethnographies are an especially good way to understand the meanings 
that participants in settings give to events and actions (for a discussion 
of the phenomenological underpinnings of ethnography, see Wilson, 1977). 
The emphasis that this approach gives to revealing the rules that parti- 
cipants use to interpret events places r. premium on bringing out these 
meanings and showing how people actively go about the business of making 
sense of their lives. A number of the criteria for ethnography help en- 
sure that participants* meanings will become apparent to the researcher. 
The use of senior investigators to conduct the research and the extensive 
time spent in the field forces them to encounter participants in the act 
of expressing and living by their own interpretations. Such barriers to 
the senior investigator's direct apprehension of the subjects* meanings 
as thoroughly prespecified data collection procedures, research assis- 
tants, and frequent escapes to the university are reduced by these cri- 
teria. Cross-cultural experience is supposed co sharpen the researcher's 
ability to perceive and respond to meanings inherent in the field by 
sensitizing him or her to variations in cultures and by inculcating the 
norm of cultural relativity. The criterion of refining the problem in 
the field encourages the researcher to encounter participants* meanings 
head-on without the conceptual blinders of preexisting theory. Finally, 
the reproduction of primary data helps the reader experience some of the 
life in the setting a little more directly and see hov^r meanings are ex- 
pressed by participants. 
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Ethnographies also illustrate the processes that take place in settings 
studied. Some of the case studies referenced above specifically focus on 
change (Firestone, Smith and Keith, Sproull et al.). Others describe the 
daily rounds of individuals and the ongoing processes that take place 
(Cusick, Wolcott). The length of data collection in ethnography—that is, 
the researcher's presence in the field for a long period of time — and the 
emphasis on direct observation ensures that process and activity will be 
seen. More cross-sectional approaches that capture smaller samples of 
behavior — e.g., questionnaire responses — are more likely to miss the dynamic 
aspects of situations. The descriptive nature of ethnographic reports 
with their wealth of primary data ensure that the nature of the process 
will be described for the reader. 

One of the special strengths of ethnography is its capacity to show 
how a vast range of contextual factors affect activity (Wilson, 1977). 
For that reason, it has been suggested that ethnography is the ultimate 
multivariate research approach. The impact of context is especially apparent 
in several of these case studies. For instance, Peshkin describes the 
relationships between a school and its community; Wolcott describes 

how a principal's behavior is conditioned by teachers, other adndnistrators, 
and his family; Firestone and Smith and Keith discuss how a variety of 
school, district, cr ..lunity, and external forces affect planned change 
processes. Once more, the several criteria that force the senior invest i- 
^tator to become immersed in a specific setting create a sensitivity to 
i<leo^;raphic contextual factors that shape activity there. 
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Finally, ethnographies can provide a wealth of evidence for conclu- 
sions and interpretations advanced. This evidence is of a special type, 
very different from that found in most quantitative studies. Quantitative 
evidence draws its claim to veracity in part from the reliability and 
replicability of data collection and analysis procedures — presumably 
another researcher can repeat the study anc' obtain similar results — and 
from the number of cases and strategic comparisons made. However, these 
strengths require the presentation of fairly abstract knowledge resulting 
from a great deal of manipulation by the researchers. 

Ethnographic evidence is more direct. Interpretations have to be 
confirmed, or at least related to the respondent's own words. The amount 
and nature of ethnographic evidence is what makes case studies »o inter- 
esting as well as believable. The use of multiple methods also lends 
credibility to ethnographic evidence. 

Ethnography also has its special set of disadvantages. Many of these 
are especially critical for an applied discipline like education which 
seeks to provide useful information to teachers, school administrators, 
and various federal, state, and local policy makers. These disadvantages 
have to ^o with costs, bulk, timeliness, unpredictability, inadequate 
attention to "manipulable variables," and generalizability . The costs of 
conducting an ethnography are considerable. Tlie researcher is in the 
field on a nearly full-time basis for at least a school year simply col- 
lecting data and may work on a part-time basis for several more years to 
produce the final report. Moreover > the costs of ethnography are increased 



12 

ERLC 



because ic requires the expensive time of a senior researcher rather than 
Che less costly time of research assistants and interviewers. 

Not all the costs are financial. For instance, there is a special 
form of response burden coming from having someone "hanging around" all 
the time and watching, although this burden may be less than might be 
expected. It is rarely complained about by subjects or mentioned by re- 
searchers. There is a definite psychic upheaval for the researcher who 
spends a great deal of time interacting with and coming to identify with 
research subjects. The marglnality of the research role creates many very 
personal questions about whether and when to intervene in a situation, 
how to judge others as individuals, and what to say to whom (Metz, 1981; 
Wacaster and Firestone, 1978). 

Another important limitation of ethnographies is their bulk; this has 
prompted numerous wry jokes over the last decade on the lines of "IJhat do 
you do with a 600 page ethnography?" However interesting a case study may 
be, most practicing educators and policy makers do not have the time to 
sic down and read one, much less several. Information has to be packaged 
in more compact forms for their use. 

A third problem with ethnographies relates to tirr.-lr.-g. Policy makers 
usually need to have results quickly because they operate in a world with 
pressing deadlines for action (Colemen, 1972). The long time required to 
complete an ethnography makes it likely that the report will be completed 
after the particular issues that contributed to funding the research have 
passed. However, insofar as the same issues tend to constantly reappear 
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in the policy world, the ethnography may speak to a new form of the original 
concern when it is completed. 

A fourth problem has to do with predictability. Funding agents, 
policy makers, and project directors being studied (among others) want 
to have some sense of what "the payoff" for an ethnography will be. In 
many cases, their needs are rather specific and short term. They may want 
to know what the outcomes of an action project are and whether It can be 
viewed as a success. Often they are involved in an advocacy activity, for 
instance hearings before a legislature, where they would like to turn to 
the ethnographer for supporting evidence (Weiss, 1972). The long time 
needed to conduct field work and write an ethnography requires that these 
information needs be put off for an extended time. Perhaps even more 
frustrating, the criterion of discovering the problem in the field precludes 
the ethnographer from giving more than the most general idea of what the 
results of the research will be in advance (Knapp. 1979). The ambiguity 
about what the payoff will be can create severe pressures on the ethno- 
grapher working in a policy research or evaluation setting. 

NIE's Experimental Schools Project was an example of one of the few 
endeavors to try to incorporate something like pure ethnography into an 
ev.lu.icion project, and the contractors conducting the ethnographies met 
with nixed success. One ethnographer (F.verhart, 1076) reports that he 
was pressured to focus on implementation outcomes and factors affecting 
th..m. The researcher saw such action as a breach ot the necessarv condi-' 
tions .or doing good ethnography. However, when on. lonrns that the tund.n- 
„,.n.v's mission vas to assess the utility of a particular implementation 
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strategy while the researcher was focusing on student peer groups, ih. 
reasons for conflict can be understood. Another contractor working for 
the same agency attempted to buffer the ethnographers from such pressures, 
but one result was decreased support for the project's ethnographic component 
and its phase out two years bef.re termination of other data collection 
activities (Firestone, 1980). 

A fifth problem is that ethnographies are not particularly stron:^ 
at identifying and analysing the effects of the manipulable variables that 
interest policy makers (Knapp, 1979). Policy makers want to know about 
the effects of factors that can be changed through legislation, funding, 
and regulation. Teachers and administrators are also interested in the 
effects of variables they can manipulate, but the variables differ. Because 
of the ethnographer's insistance that research issues be discovered in the 
field, the policy maker has little guarantee that manipulable variables 
will be addressed. The case of the ethnographer who studies junior high 
school peer groups when the funding agent wanted to know how to implement 
"comprehensive" change could give pause to any agoncv considerxn. funding 
ethnographic research. Finally, even where the ethnographer addresses 
the general issue of interest to policy makers, the tendencv to s^rress 
context (Wilson. 1977) often means that manipulable variables receive 
relatively little attention. Thus, where Firestone (1980) did study the 
implementation process that interested the agency fundin, his vork, his 
i„.,„o,-sion in a school district encouraged him to hi.hligh, H.i.ations Co 

>.npa'-t of mnnipulahle variables created by local authoritv structures . 
and conni<.-.s rather than th. impact of funding strategies and regulation. 



Yet, this study does illustrate how attempts to promote a particular kind 
of change from afar are undercut by local contextual factors and change 
processes.: Thus the weakness of policy variables is a finding as wel^ as 
a result of the methodology. 

The final limitation has to do with the generalizability of an ethno- 
graphic account. Once a particular case has been described in great detail, 
how does one know if interpretations are applicable in other situations? 
Policy makers are interested in general applications of findings while 
practitioners want to know how a finding from the study setting will apply 
to their particular situations. There is some tendency to believe more 
easily in the generalizability of large sample, quantitative research 
simply because of the number of cases included (Mulhauser, 1976). This 
tendency may be misplaced, of course, when it is not clear how the sample 
studied represents the larger population to which one wishes to generalize. 
This is often the case with convenience samples from a neighboring region 
or studies of special projects. Kennedy (1979) and Stake (1978) suggest 
that Che task of generalizing from an ethnography or, more properly, npplyin. 
findings should be left to the reader. The person whc uses the extensive 
description in the ethnography should estimate the applicability of various 
aspects of it to his or her own situation. 
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Varieties of Qualitative Method s 
The ethnographic approach to research accentuates a number of the 
characteristics of general qualitative research. There are many situa- 
tions where qualitative resc-arch is useful and appropriate while pure 
ethnography is not. One of the most exciting trends in the application 
of qualitative research to education and to other applied fields, includ- 
ing evaluation and policy science, has been the willingness to innovate 
with the use of qualitative methods. Researchers have borrowed from the 
existing array of qualitative techniques in more disciplinary research. 
But they are also developing new approaches, sometimes by adapting quali- 
tative techniques and sometimes by borrowing quantitative techniques. 
The result is considerable ferment and variety. This section describes 
and helps organize various data collection and analysis strategies that 
have been used in qualitative inquiry. 

There is a tendency to think of data collection and analysis as 
separate stages in a linear research process, but that is not the preferred 
mode of organizing qualitative projects. When the researcher is used as 
the instrument, incoming data— the very experience of observing and inter- 
viewing — can suggest new concepts and relationships (Geer, 1968), More- 
over, ongoing data analysis can guide data collection by suggesting new 
area^ to investigate and hypothesized relationships to examine, Claser 
and Strauss* (1967) constant comparative method for data analysis is 
partly a way to guide data collection. 



17 

ERiC 2 i 



Data Collection 

Data collection strategies can vary in at least five ways: the num- 
ber of sites included in the study, the extent of researcher immersion 
in the site, the extent to which data collection activities are struc- 
tured in advance, the extent to which researchers have generalizing or 
particularizing frameworks for observation, and reporting strat2gies. 

Number of sites . The classic ethnographic product is a case study 
of a single group. Anthropology does have a comparative science of 
ethnology, but its practitioners have historically relied on the published 
accounts of ethnographers supplemented with artifacts those individuals 
brought back (for example see Benedict (1934)). 

There are numerous examples of multisite qualitative studies in the 
educational literature. At one extreme are the single researcher's 
projects with a limited number of sites such as Metz' (1978) examination 
of authority structures in two desegregated junior high schools and 
Bossert's (1979) examinations cf task structures and social relationships 
in four classrooms. At the other end are the large, federally-funded 
projects with numerous sites. These usually rely on teams of researchers. 
For instance, the Rand Change Agent Study included Ik case tJtudies of 
federally funded innovations (Greenwood, Mann, and McLaughlin, 1975). 

Most of the large studies combine qualitative and quantitative tech- 
niques in varying degrees. For instance, the Rand study had a substantial 
survey component. The exception to this rule, and perhaps the extreme 
rase of larj>e-scale qualitative research, is a recently completed project 
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by Ray Rist that collected and synthesized participant observation data 
from 60 youth employment training programs through a series of interim 
reports to Congress and the relevant federal agencies (Rist, 1981). 

Multisite studies have at least two advantages. First, several sites 
help to overcome some of the generalizability problem. Generally, as pat- 
terns are repeated in several locations, the reader can gain confidence 
that findings are not a fluke resulting from some special characteristic 
of Che site. Second, there is the possibility of making comparisons. For 
instance, Bossert's (1979) argument that the modal pattern of classroom 
activity affects the friendship choices of students and teacher-pupil 
interaction is based on the contrast between two pairs of classrooms. 

Extent of immersion. Generally, the more time the researcher spends 
in the sice, the better he or she comes to understand it and is able to 
draw lessons from it, for at least three reasons. First, the more time 
a researcher spends with a group, the more likely he or she is to accurately 
interpret their meanings (Bruyn, 1966). Second, spending sufficient time 
In a setting allows a researcher tc observe the dynamics of change in it 
,ind reduces the likelihood that observations will be conducted at an 
unrepresencative time (Bogdan and Taylor, 1975). Finally, time is Important 
because it allows a researcher to test his or her conclusions many times 
in many ways (Becker, 1970). Here again, ethnography with its criterion 
of year-long immersion in a single setting represents a requirement Chat 
educational studies frequently cannot meet. The requirement for nenrlv 
tull-r.ime immersion is expensive and time consuming and effectivclv pre- 
cludes cross-site comparison in a one-person proiect. 



There are two typical alternatives to the ethnographic form of high 
immersion. One is the short-term site visit. Usually, a team of researchers 
will visit a site for as few as two or three days. The intensive site visit 
is often used in large-scale national studies where travel is problematic. 
The short-term site visit has been an important data collection strategy 
for national studies of the politics of school desegregation (Grain, 1968). 
district factors promoting educational change (Herman and Mclaughlin, 1979), 
and the status of science education (Stake and Easley, 1978). 

A number of design decisions can enhance the validity and usefulness 
of short-term site visits. First, a team of researchers can "cover** the 
territory faster and can check each other's impressions. V?ith adequate 
coordination, disagreements about facts or conclusions can be resolved 
through further questions before leaving the site. Second, the researchers 
can be given a rather explicit idea of what they are looking for. KTien 
time is short, more focused interviewing and observation are necessarv. 
This often means that experienced researchers with previous understandings 

the kind of settings and issues being researched are preferred (McLaunlil ni , 
1981). Finally, it is extremely useful to be able to make at least two 
visits (Yin and Gwaltney, 1981). That way, impressions formed after i he 
firf;t trip can be clarified and tested with new data. 

The second alternative is an ongoing series of intermittent visits. 
For instance, in one three-year study of educational change, researchers 
visited several schools on a once-a-i» >ek basis (Corhett, H)80; Dawson, 
1981). Ihe overall elapsed time for data collection wa.- lon\?.er than for 
some ethnograpliies, hut the number of hours spent in the schools was some- 
v/hat less. 
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Intermittent visits are not useful when sites are beyond driving 
distance; hui for sites in a circumscribed area, they have at least two 
advantages. First, it is possible to build rapport with respondents in 
a way that is not possible in a short-term site visit. Researchers get 
to know the people in the school and vice versa. Thus, relationships 
develop which encourage respondents to give personal accounts of attitudes 
and orientations as well as interpretations of events. There is more 
opportunity to learn about facets of .school life that are normally kept 
from strangers. Second, there are numerous opportunities tc observe a 
variety of settings over an extended period of time. In the short-term 
site visit the researcher is only privv to whatever happens on the few days 
he or she is there and must relv on retrosoective interviews alone to get 
a sense of history. During intermittent site visits, the researcher has 
opportunities to observe classrooms, teachers' lounges, playgrounds and 
parents' meetings. In addition to observation in these recurrent settings. 
It is possible—sometimes with the help of a cooperative informant—to 
arran^;e Co be on the scene for critical events such as special board or 
staff meetings. The researcher is also able to observe change in the set- 
tinv;. Such observation allows che intermittent site visit approach to be 
ris effective as che ethnography in capturing the process. 

(ither things being equa^ , the more time spent at a site, the better, 
up to a point. However, that point has not been well established. Often, 
the tlori^ion to terminate data collection is dictated bv constraints of 
tim«' .ind money. Huha (1^U8) has suggested some substantive guidelines for 
wlu'ii ro st(»p iht' most open-endod forms of data collection. These include 



exhaustion of data sources, saturation—that is, the point where categories 
are well understood and data seem redundant— the emerRence of regularities 
CO the investigator, and c verextension or the point at which the investi- 
gation *begins to carry into geographic areas removed from the issue or 
concern of Interest . 

Extent of structuring . The ways of categorizing the amount of structure 
in a project are not well developed. The end points are clear enough, 
however. On the one hand, there is the situation with seemingly no struc- 
ture; the researcher sets out with an idea, observes without plan, analyzes 
the field notes, and writes a report. The other extreme is the study where 
a complete research design is developed even before data collection begins, 
the classical quantitative study. Neither ot these extremes is found fre- 
quently in qualitative research. Instead, two interim structuring devices 
are used in various combinations. 

The first is the use of instrumentation and sampling plans analagous 
to those found in quantitative research. These are usually introduced as 
progressive focusing helps define the research problem and data needs. 
The instruments used in qualitative research ensure that data on prespeci- 
fied topics will be collected systematically from a -jample of individuals 
or settings— i.e. , the amount of missing data will be reduced. However, 
these instruments vary substantially in the extent to which they an pre- 
structiired. The least structured instrument might he the site visit guide 
which is essentially a list of topics, issues, or questions to guide obser- 
vation and interviewing. A more structured instrument is the open-ende.i 
interview guide which specifies the questions the interviewer asks of a 
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respondent (although not necessarily the exact words) and indicates areas 
to be probed. The most structured instrument is the precoded questionnaire 
or interview guide where the interviewer must repeat the questions as they 
are written and check off the appropriate response (Gordon, 1975). The 
analog for observation is the observation guide which specifies in advance 
the categories of behavior to rate or count (Rosenshine and Furst, 1973). 

o-ampling plans serve two purposes. First, they allow the researcher 
to collect information from a few people and generalize to a larger popu- 
lation, thereby minimizing time and effort. Second, they help overcome 
bias in the selection of informants. A qualitative researcher often relies 
heavily on key informants to help interpret what he or she sees, to explain 
the rules and norms of the group, and to provide other information. However, 
res£-:rchers are often biased in selecting informants. For instance, they 
are likely to select elite members rather than those from the rank-and-file 
(Sieber, 1973). 

Random sampling is rare in qualitative research. It is intended to 
enhance generalizability by overcoming unknown sources of bias. At the 
same time, it makes use of the laws of probability. By contrast, quali- 
tative researchers have a great deal of information- about their cases which 
can be used to anticipate sources of bias and to guide sampling. As a 
result, they can rely on several other sampling strategies. Represen- 
tative sampling occurs when sites or informants are selected because 
they are considered typical of a larger population. Another kind of 
sampling Is purposive, strategic or theoretical sampling. In this 
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approach small numbers of informants are selected to vary on critical 
dimensions. Bias can be reduced by introducing variation on sources of 
bias which the researcher knows from previous observation. At the same 
time, the researcher creates contrasts that get ir'ormation that helps 
illuminate an aspect of the research problem. Snowball sampling involves 
using informants to identify others who should be interviewed (Denzin, 
1970) . 

These sampling designs can be simple but they may be rather complex. 
McClintock, Brannon, and Maynard-Moody (1979) describe a number of sampling 
designs used within single case studies that simultaneously stratify on 
informant position and the kind of event selected for analysis. Researchers 
also select extreme cases, deviant cases, or cases where one has a strong 
basis for predicting that something will or will not be present for special 
attention. 

The other structuring device comes through the researcher's own 
internalized theoretical sensitivity. Such sensitivity is useful because 
it heightens the researcher's awareness to some phenomena, but it tends 
to blunt consciousness of others. Theoretical blinders may be inappro- 
priate in the early stages of a qualitative study. However, as the work 
procuresses and the researcher develops an increased sense of what the 
important findings might be, such sensitivity becomes essential if closure 
is to be achieved and the report written. Theoretical sensitivity comes 
from several sources. One is previous formal training. For instance, in 
listening to the same discussion by an informant, an organizational socio- 
lonLst may pick out data relevant to organizational goals and structural 
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patterns in a school; a social psychologist may hear more about norms, 
roles, and values; and a linguistic anthropologist may hear more about 
language patterns and thought processes. 

A second source of theoretical sensitivity is the ongoing process of 
data collection and analysis within a project. Geer (1967) describes 
how the process of rereading field notes and writing memos about impres- 
sions during the first days in the field helps identify concepts to be 
pursued during later work in a study of college students. Miles (1979:594) 
describes how a collective project where staff field workers and others 
developed, implemented, and discarded a number of systems for coding raw 
field notes finally led to "a common language of concepts which found their 
way into the general (research) framework, and guided further data analy- 
sis in less formal modes." 

A third source of theoretical sensitivity that is most relevant to 
larger projects is intentional training. Many projects that use numerous 
field workers, especially for short-term site visits, provide extensive 
training for those individuals. Such training often deals at length with 
the research issues of the project and gives visitors an idea of what to 
look for (McLaughlin, 1981). This training often accompanies the use of 
site visit guides. 

The amount of structure in a qualitative study will depend on a num- 
ber of factors including the number of sites, the length of the study, 
and the extent to which research problems are prespecif ied . Comparative 
cross-site studies, studies of shorter duration and studies wich pte- 
specified problems typically start with more structure. 
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The researcher's framework for observing , V/lien the researcher does 
most of the observation, variation in personal characteristics will affect 
the kind of observations made and conclusions drawn. An especially impor-- 
tant characteristic is whether the researcher brings a generalizing cog- 
nitive framework or one that parr\cularizes to the study, 

A generalizing framework helps the researcher identify some broader 
import or special relevance of the phenomena observed. It can also help 
identify what is to be observed by presenting a contrast with what has 
been learned previously — hence, Kluckhon*s comment about fish not dis- 
covering water. There are at least three ways to develop such a frame- 
work. The anthropologist's preferred method is cross-cultural experience 
(see above) . Cross-cultural experience provides first-hand knowledge of 
other ways of doing things and, in conjunction with proper training, a 
relativistic attitude about different modes of behavior that permits one 
to withhold judgment until research is completed, or even later. The 
point is not so much a change in countries, however, as contrasting 
experience. Placing an urban, university researcher in a rural school 
district or a white researcher in a black ghetto should have the same 
effect on a somewhat smaller scale. The c'isadvantage of such preparation 
is that researchers may spend a great deal of time discovering the obvious 
before any useful contribution to the knowledge of others can be made. 
For example, Freilich (1970) presents an example of one anthropolo>;ist 
required to spend two years doing an ethnography where most of the first 
vear was spent learning the language. 
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A second way to obtain a generalizing framework is to become well 
versed in a body of sociological, anthropological, or psychological 
theory. Wolcott (1975) stresses the importance of a strong background 
in cultural anthropology for ethnographers, but a similar effect would be 
obtained through immersion' in any of several disciplines. Such immersion 
gives the researcher a body of concepts (not to mention descriptions and 
findings from previous studies) to use to make sense out of the research 
situation. 

Finally, especially when the field work is intended to address 
issues of practice, a thorough grounding of the problems facing an applied 
field can provide a generalizing framework. Such a grounding sensitizes 
the researcher to issues and keeps him or her on the lookout for possible 
solutions. However, it can have the opposite effect, creating a set of 
blinders by convincing the researcher that what is observed is either 
impossible or immoral for the occupational group of interest before the 
issue can be fully explored. 

A particularizing framework comes from knowing the specific situation 
very well. Campbell (1974) argues that perple who are very familiar with 
the setting, especially the actual research subjects, have a substantial 
body of practical knowledge about those settings. Some of it is tacit and 
difficult to verbalize. Thus, the fish may not discover water, but it can 
swim better than an anthropologist. Other parts are easily articulated. 
Campbell argues for incorporating people from the setting into the re- 
search team in some role as more than subjects because thev can bring 
this tacit, practical knowledge to the research effort. It seems prob- 
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able chat where research is conducted solely by members of the setting 
of interest — e.g., a teacher or a school district evaluator—these people 
may have trouble obtaining the generalizing framework needed to create a 
useful study. Moreover, the social system studied may constrain what 
they are willing to say because fears of reprisals or ostracism or 
what they actually do see. 

Much of the better qualitative research being conducted today is done 
by individuals with a strong grounding in a social science discipline 
and general familiarity with educational issues and problems but without 
cross-cultural experience. These people have generalized frameworks but 
are far from naive about the settings they study. There have been some 
successful experiments with collaborations between professional researchers 
and members of the system studied. For instance, Louis Smith, a university 
researcher, and William Geoffrey, an elementary school teacher, conducted 
an ethnography of Geoffrey's classroom (Smith and Geoffrey, 1967). 

Reporting requir^ents. Decisions about who will receive reports make 
a substantial difference in what information will be available to the re- 
searcher. The classic roles of the anthropological ethnographer or 
the sociological participant observer were determined in large measure by 
the expectation that these individuals would not provide feedback to the 
social system studied. Pelto (1970:220) describes the ethnographer's 
role as one of "interested neutrality*' meaning that the researcher's per- 
sonal opinions or research conclusions are not fed back to the site al- 
though they are shared with other researchers. He suggests that the 
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researcher "is different (from the research subjects) because his core 
prestige ultimately rests on membership in another, socially distant 
society; he is neutral in the local competitive scene.. .and he offers 
social rewards in exchange for information/* The ethnographer's neutrality 
stems from the expectation that ethnographic reports will not filter back 
to the society studied but will circulate only among the special social 
system of academics. He can be trusted because he is not part of the 
social system. Under these conditions respondents can divulge information 
that would otherwise be embarassing or even threatening to a social posi- 
tion or career. 

The modern qualitative researcher in education rarely operates in a 
setting where the stance of interested neutrality can be maintained. With 
the rise of modern specialties of evaluation and policy research, there 
is a growing expectation that research results will be fed back to the 
subject's social system. Research recipients might include the subjects 
themselves, their superiors, their funding agencies, those writing legis- 
lation governing their worklife, or simply colleagues whose respect is 
valued. 

The feedback of research results both opens and closes doors for the 
researcher, and it affects the kinds of information that respondents will 
provide. Doors are opened when respondents believe that they will obtain 
feedback that will help them do their work better, when they will obtain 
positive publicity, and when they can tell "their story" to some powerful 
decision maker who would not otherwise hear it. Doors are closed when 
respondents believe they will get negative publicity and when they believe 
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that the results of the study will lead to decisions with negative conse- 
quences for them. 

Firestone (1980) had to deal with these issues in his study of a 
rural school system where there was a generally accepted belief that he 
was reporting back to the fundin? agency on the implementation and out- 
comes of a grant. (The belief was generally correct, but respondents 
did not understand that fclie report would not be submitted until after the 
project was completed.) The project was a source of major conflict between 
project leadership, especially the district superintendent, and a 
substantial group of teachers who opposed the project. Both groups were 
self-serving in the information they provided. However, since the project 
leaders controlled formal reporting to the funding agency, they tended to 
be reticent. The superintendent in particular was difficult (although 
not impossible) to approach and emphasized the positive. Opposition 
teachers who had limited opportunities for direct communication with the 
funding agency were much more approachable and emphasized the negative. 

In sunt, researchers can rarely adopt the older stance of interested 
neutrality because they are usually in some sense part of the social sys- 
tem they are studying. Their reports will have consequences for the re- 
spondents. Yet, the classic problems of obtaining the respondent's trust 
remain; without that trust it will be difficult, if not impossible, to 
obtain the valid data with minimal bias needed for a successful report. 
Still, with knowledge of the specific situation, the researcher can make 
decisions — about the audience for reports, the timing of those reports. 
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and the extent to which specific individuals and institutions are kept 
anonymous — that can increase 'cb.e liklihood of developing and maintaining 
the trust necessary to obtaining high quality data. 

Data Ana lysis 

The topic of data analysis in qualitative research is one of the 
most underexamined in the field. Miles (1979) quotes Sam Sieber as say- 
ing that most textbooks on qualitative research devote less than 10% of 
their content to analysis. This may be the area where current work in 
education has the most to offer to fields with a longer tradition of 
usinu these techniques. 

Data analysis serves at least two major functions. The first is 
the interpretation of research information. Through data analysis, 
researchers aggregate and synthesize masses of data 'into a meaningful 
depiction of a situation. They also identify new concepts and relation- 
ships suggested by the data. Second, data analysis serves a verification 
function at several levels. For instance, researchers may verify facts, 
interpretations, the usefulness of specific concepts or variables, and 
relationships among concepts or variables. 

Although there is a tendency to think of interpretation preceding 
verification, these functions are typically accomplished through an iter- 
ative process in qualitative research. Researchers will begin to develop 
an interpretation and then discard or modify it as contradictory data are 
discovered. A new or revised interpretation will be developed and then 
subjected lo a more formal test only to be again discarded or amended. 
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A variety of techniques or approaches are available for both inter- 
pretation and verification of qualitative data. While these techniques 
are quite diverse and can be combined in many "different ways, it is useful 
to sort them into three sets—intuition, procedural techniques, and » 
intersubiective techniques. The bulk of this discussion will examine 
how each set of techniques can be applied to the tasks of interpretation 
and verification. A special section will examine applications of each of 
these techniijues to cross-site comparison because that activity has 
special requirements. 

General analysis strategies . Individual intuition. is at the heart 
of the interpretation of qualitative research data, but its use is diffi- 
cult to describe. Basically, the researcher becomes very familiar with 
the research phenomenon and his or her materials about it — the actual 
field setting, notes and memories of interviews and observations of events, 
and artifacts. This material is placed against preexisting experiences, 
theories, and formulations of problems in a process that is often sublim- 
inal. Through immersion and contemplation, findings emerge. The process 
is often marked by numerous interim memos through which the researcher 
records and refines observations. 

Intuitive analysis can produce an interpretation that is very satis- 
fying to the researcher; and in the hands of a skilled writer, the result 
can be extremely convincing to the reader. However, intuitive analysis 
creates a serious problem of credibility. Speaking from the viewpoint of 
n self-critica] qualitative researcher. Miles (1979:590) asks, "how can 
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we be sure that an 'earthy,' 'undeniable,' 'serendipitous' finding is not, 

in fact, wrong." Interpretations must be verified; those developed through 

intuitive analysis particularly need to be verified through techniques 

that can be described to readers. 

Intuitive techniques can be used for verification by generating 

predictions and hypotheses from the interpretation and checking them 

against existing field notes and through further data collection. 

According to Campbell (1975:181-182): 

In a case study done by an alert social scientist who has 
thorough local acquaintance, the theory he uses to explain 
the focal difference also generates predictions or expecta- 
tions on dozens of other aspects of the culture, and he does 
not retain the theory unless most of these are confirmed. 
In some sense, he has tested the theory with degrees of free- 
dom coming from the multiple implications of any one theory. 
The process is a kind of pattern-matching in which there are 
many aspects of the pattern demanded by theory that are avail- 
able for matching with his observations... . 

This pattern matching is an important source of rigor and verification in 
case study analysis. Researchers do in fact reject interpretations that 
do not fit enough of the facts. However, the process may be relatively 
ineffective at improving credibility if it is invisible to the reader as 
is usually the case. If credibility is to be improved in the future, the 
intuitive process and some illustrations of its application during a 
study should be described to the reader as part of methodology statements. 

Procedural techniques are essentially rule-bound. The researcher 
follows a procedure to its logical end before drawing a conclusion or 
verifying it. Such procedures are used primarily to accept or reject 
data or interpretations; however, the procedures can also be used to de- 
velop interpretations. In fact, the constant comparative method developed 
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by Glaser and Strauss (1967) is intended to generate new concepts. Many 
procedural techniques are quantitative in that they require some form of 
coding and counting. However,^ while rule-bound, they vary in the extent 
to which they are mechanistic. Many require a great deal of judgment at 
some points, particularly when data are coded or when decisions about the 
adequacy of data are made. 

The major procedural technique which is used for generating interpre- 
tations is the constant comparative method for discovering grounded theory 
(Glaser and Strauss, 1967),, As described by its inventors, this method 
has four parts, 

• Comparing incidents applicable to a category . The researcher 
compares the new incident to already coded incidents while 
the coding is in process in order to identify relevant 
dimensions of variation in the category. As new dimensions 
become apparent, they are recorded in memos. 

• Integrating categories and their properties . As coding 
continues the unit coded changes from the incident to the 
property or dimension. This process helps identify the most 
important explanatory and descriptive categories and develop 
more abstract categories into which more concrete categories 
are placed, 

• Delimiting the theory . Through further review of- field 
notes, coding schemes, and memos, the researcher creates a 
smaller set of more general concepts. This step increases 
both the parsimony and generalizability of the developing 
theory , 

• Writing the theory . With a well organized data set, memo 
file and theory, the researcher can quickly write up the 
results , 

Many procedural techniques are available for verifying data (Dawson, 
1980), They vary in complexity depending on the issues involved and the 
inventiveness of the researchers. Perhaps the simplest is a rule that 
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prohibits the researcher from reporting a fact that does not have multiple 
confirmations. For instance, Greenstone and Peterson (1973) report that 
their accounts of Community Action Programs in American cities included 
facts only that had been verified by three or more witnesses. 

The existence of analytic categories may require more complex veri- 
fication that takes into account not only the number of times an event 
or statement occurs but also the context. For instance, Becker and Geer 
(1960) report on a procedure for evaluating data indicating the existence 
of a "perspective" or worldview among a group. This procedure codes 
statements in fielf' notes according to whether they are volunteered or 
made in response to questions and whether they are made with the inter- 
viewer alone or in front of peers. It also codes activities according to 
whether they take place in individual or group settings. Rules of thumb 
for reading these tables suggest that where more statements are volun- 
teered, where more statements are made in a group context, and where 
there is a balance of statements and activities, then the assurance that 
a perspective actually exists is increased. 

Another way to verify the existence of an analytic category is to 
"triangulate" or look for convergence across methodologies (Webb et al., 
1966). The assumption behind this approach is that different methodolo- 
gies have compensatory strengths and weaknesses. Where several method- 
ologies lead to the same conclusion, the researcher's confidence in that 
conclusion is increased substantially. Triangulation may be especially 
interesting to qualitative researchers because the methods used together 
are usually some combination survey and qualitative observation and 
interviewing. 
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While triangulation is much discussed, two problems face tbjse 
attempting to implement it* First, there is more art than science to 
knowing when two methods in fact present confirmatory evidence. Second, 
when different methods conflic, there are few guidelines for determining 
which is more correct. Divergence may actually reflect opportunities to 
refine interpretations, however, jfick (19^9^ describes a situation where 
survey ana observational data presented contradictory evidence about who 
was most distressed by an impending plant merger. Further interviewing 
led to the discovery that those engaging in the observed behavior—check- 
ing the archives for information on similar events in the past — were in 
fact engaging in behavior that helped reduce stress, thereby leading to 
the discovery of a new kind of coping mechanism. 

Procedures could be developed for other purposes, such as showing 

the distribution of phenomena and providing evidence for causal relation- 
. 1 

ships, but less work has been done in these areas. Once data are coded, 
there is a whole range of descriptive statistics that can be used to show 
distributions."^ The simpler ones, like percentages and differences, are 
often most useful. There are at least two approaches for showing causal 
relationships. One is quantitative — based on using correlational tech- 
niques, anything from scatter diagrams to cross-lagged correlations, 
depending on the quality of the data— to show association. There the body 
of procedures in the field of statistics apply. The other, which may be 
more properly qualitative, is the description of process—that is, showing 

I 

It is important to point out here that such procedures can be used only 
when research phenomena are rated in some manner, such as high to low 
or good to poor, not when their presence or absence is noted. Uheu 
research procedures are not highly structured, the presence or absence 
of data about something does not pecessarily represent its actual 
presence or absence in the setting. 



the mediating activities through which an event or orientation comes into 
existence. However, this approach now depends more on intuition than any 

established procedure. 

Intersubjective techniques generally include strategies which involve 
interaction among a team of researchers or between researchers and setting 
participants. Intersubjective techniques are probably used more frequently 
for verification than interpretation, particularly when they involve in- 
teraction between researchers and setting participants, but can be used 
for interpretation as well. Intersubjective approaches are in fact 
another form of triangulation. Whether for interpretation or verifica- 
tion, they involve using multiple sources of information. 

Interpretations can be generated through collaboration among re- 
searchers. Researchers can share their impressions of events in a setting 
and develop interpretations which have been refined considerably during 
development. Since teams of researchers generally study multiple sites, 
this technique will be described more fully below, in the discussion of 
intersubjective techniques in cross-site analysis. Teams of re- 
searchers do occasionally study single sites, but there have been few 
discussions of how such teams operate. Examples of team research include 
several studies where Louis Smith was the principal investigator (Smith , 
and Geoffrey, 1968; Smith & Keith, 1971; Smith and Pohland, 1974) and the 
studies by Becker and associates of a medical school (Becker, Geer, 
Hughes, and Strauss, 1961) and of undergraduate students (Becker, Geer, 
and Hughes, 1968) . 
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As noted above, interaction between researchers and setting 
participants during <"ita analysis is more likely to focus on verification 
than interpretation, but the latter does occur. Researchers sometimes 
viiscuss research findings witn participants and ask for their perceptions 
of why things occurred. Resear.:hers also sometimes informally toss out 
tentative interpretations to research participants hoping to stimulate 
them to suggest major or minor refinements. 

Verification through in tersubj active techniques generally involve 
taking the strategies described above one step further. Researchers 
explicitly test their interpretations with other researchers or with 
setting participants. Researchers may write memos or drafts of reports 
and ask others to react to them. Interpretations are considered much 
more likely to be valid if they have been confirmed by people with other 
perspectives, people such as other researchers or setting participants. 
It is becoming more common for researchers to present drafts of reports 
to respondents during or after a study, Yin and Gwaltney (1981) specifi- 
cally designed their site visits in two waves to that they could test 
their interpretations with research subjects. The first visit was to 
learn about the site. A major agenda item for the second visit was to 
present site staff with a draft case study and obtain a critique that 
was used to correct facts and revise interpretations. 

Our own experience suggests that this approach may set limits to 
t \e dtiLn that ran be reported. As Becker (196^0 points out, research sub- 
jects are likely to ask for changes that protect their images of themselve 



and their schools, organizations, or communities. Moreover, when research 
reports are submitted to funding agencies, survival and growth as well 
as self-esteem may be at issue. We are studying how work groups in our 
own organization provide technical assistance to schools, and our reports 
present descriptions of the work groups as well as the schools. All of 
the reports are presented to these work groups before being released to the 
public — including the major funding agency for our organization. VMle 
these reviews have definitely provided new data on the events that took 
place in schools, we have also been criticized for deviating from the work 
groups' own interpretation of events or for presenting interpretations that 
could be viewed as negative evaluations of the work groups' efforts. After 
several review meetings, it became apparent that certain findings, whether 
true or not, could not be presented. In sum, review bv subjects is a useful 
way to validate findings and to protect research subjects, but it may also 
limit what can be present^.d in a report. 

Special cross-site strategies . The techniques described above may be 
useful in multisite studies to verify facts and develop and verify inter- 
pretations; however, multisite studies may also require special tech- 
niques for cross-site comparisons. Such comparisons can be made from 
three different data bases: the researcher's experience in several sites, 
field notes and other records collected from different sites, and completed 
case studies. In fact, some of the more interesting cross-site analyses 
are actually secondary analyses of studies written by others (Dunn and 
Swierczeck, 1977; Yin and Heald, 1975). 
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As in general analysis strategies, there is an important place for 
intuition in the interpretation of cross-site data, and the way such in- 
tuition works is poorly understood. The researcher generally becomes 
very familiar with the research site, immerses himself or herself in the 
research data, and an interpretation emerges. The researcher may consider 
several alternative interpretations before settling with the one that 
seems to work best. Sometimes a researcher may work concurrently in several 
sites (e.g., Metz, 1978) and an interpretation may emerge from a comparison 
of similarities and differences across sites. Alternatively, the researcher 
may work with completed case studies, some or all of which may have been 
conducted by others. For instance, Clark (1970) used three case studies 
of distinctive colleges to develop the concept of the organizational saga — 
a value-laden conception of organizational history that guides and limits 
direction of future orgainzational change. 

Yin (1981) describes an intuitive strategy which he labels the case- 
comparison approach. The hallmark of this approach is working outward 
from the single case. First, the researcher develops an explanation for 
the outcomes in a single case based on all the data from that case. Next 
the researcher seeks to apply the same explanation to a second case. Mod- 
ifications are made to fit the specifics of the s^iven situation. The 
researcher continues working outward from one case to the next until an 
explanation is developed that applies to all cases* The art of the matter 
is knowing the acceptable limits of modification of the original explana- 
tion; but if well done, this approach should lead to grounded g^cneral ex- 
planations with appropriate limitations because of ocher 'Variables/' 
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This approach is an interesting example of how interpretation and verifi- 
cation are combined iteratively in a single technique. Each time an 
interpretation is applied to a new site it is, to some extent, tested. 
The approach has been applied in an interesting manner to understand how 
networks centered on Educational Service Agencies contribute to knowledge 
use in schools (Yin and Gwaltney, 1981). It is not always effective, 
however. For instance. Grain (1968) reports Chat explanations generated 
separately from separate case studies of northern urban school desegrega- 
tion could not be applied usefully to the set of cases. Instead, he was 
forced to turn to the sort of procedural techniques described below. We 
do not yet know under what conditions working outward from single cases 
is the most effective way to proceed. 

A procedural technique which can be used to refine and verify cross- 
site interpretations is the constant comparative method of analysis devel- 
oped by Glaser and Strauss (1967) which was described earlier. Tl^e method 
can be combined with data collection and analysis driven by theoretical 
sampling—Chat is, including new groups in a study in order to 
progressively refine an interpretation or theoretical analysis. As a 
theory emerges, researchers decide what kinds of data need to be col- 
lected next and what group would be likely to provide data which would 
be most informative to the theory. The group is then studied and appro- 
priate comparisons are made. This process is continued until the data 
which are collected no longer produce information that is useful to the 
further development of the theory; Glaser and Strauss refer to this as 
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the point of "theoretical saturation." While they emphasize that the 
strategies should be used for generating rather than testing theory, 
they do have sufficient confidence in grounded theory to recommend that 
it be used without waiting until it has been tested further. And, the 
fact that new groups are ideatifled and studied to the point of satura- 
tion means that ir.r.erpretations have been tested at least as thoroughly 
as they would have through most other verlficPtion strategies. 

Most procedural techniques for cross-site comparison tend to stress 
verification more than interpretation. Th^y also tend to be more quanti- 
tative than the Glaser and Strauss approach. One such technique is to 
present distributions of events in various sices. For instance, Bossert 
(1979) shows the relationship between task structures and social rela- 
tionships in classrooms through a two-step process,. First, he shows that 
different classrooms have different task structures by presenting dis- 
tributions of four kinds of activities in the four classrooms: recita- 
tion (all group question *ind answer), class task (all students working on 
the same task at their desks), multitask (students working on several 
tasks), and free time. Next, he presents rates of disciplinary activity 
and sociometric data by classroom in order to show that classes with dif- 
ferent rates of recitation have different teacher-student and student- 
student relationships. 

Another approach, which has many variations, is to code sites on a 
set of variables and look for allocations among variables. Yin and Heald 
(1975) have labelled this approach the case survey method. Firestone and 
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Corbett (1981) were a.ong the fa, to do this directly fro» field experi- 
ence end field notes. First, as a .eans of data reduction. 42 descriptive 
questions were asked about events In eleven school tea.s planning for 
educational change. When for.s for answering these questions „ere com- 
pleted, illustrative quotes were also entered along with page numbers in 
field notes and assessments of data quality. Second, the sheets were used 
.0 Identify nine variables related to change, and teams were rated as 
high, medium, and low on each. Pinally. blvariate scatter diagrams were 
constructed. These suggested that change outcomes were more a result of 
school factors than of outside interventions. Similar approaches have been 
used with completed case studies. For instance, .in and Heald (1975) applied 
much the same logic to 269 case studies of governmental decentralisation in 
in urban areas. 

several intersubjective techniques have been used in the interpreta- 
tion of data from multiple sites. One approach involves using a team of 
.esearchers working together. For instance. Stake and Basley (1978) used 

*.u ^^-^hiit? nf Qrience education in ten 
a team of researchers to examine the status of science 

school districts across the country. Ten case studies were conducted in 

a staggered schedule so that later studies could inform and confirm findings 

T .HH^^ion a research team consisting of the staff at 
from earlier ones. In addition, a researu 

the central location and some of the case study researchers met to discuss 
research findings. These discussions focused on both within-site and cross- 
.tte issues. The data base used In the discussions included impressions from 
sit. visits and field work, records of interviews and other field experiences, 
and completed case studies. Through the meetings, the staff refined a list of 
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ijsues and problems that had been identified when the study proposal was 
written (Stake, 1981), developed a shared understanding of those issues, 
and identified topics For subsequent data collection. Through this process, 
the research team was able to develop and substantiate a multisite analysis , 
of science education that was presented in the project's final report. 

Another approach is to develop multiple, independent cross-site 
syntheses. Herriott and Gross (1979) followed this approach. They pre- 
sented synthesizers with five detailed case studies written to a common 
format. Each synthesizer was asked to review the cases and draw implica- 
tions for a particular user group — school administrators, federal program 
designers, federal program implementors , and trainers of school administra- 
tors among others. Finally, the editors analyzed the total opus drawing 
on both the original case studies and the first set of syntheses. The 
report of this effort presents the case studies, the several syntheses, 
and the overall analysis. The reader working across the report can 
identify commonalities in the various syntheses and analyses, identify 
"findings" of interest to a particular audience, and draw his or her own 
conclusions from the original "data." 

The intersubjective strategies described here for interpretation in- 
volve so much collaboration among so many people who are familiar with 
the sites studied that a researcher can be fairly confident that the 
data and interpretations are valid. However, there may also be a stage 
at which an interpretation that has been generated collaboratively is then 
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very deliberately tested among the team of researchers. For example, 
the interpretation may be put in writing and submitted to all members of 
the team for their reactions. They will particularly compare the interpre- 
tations with their impressions, and perhaps with their field data, of what 
occurred in the sites for which they were responsible. While interpreta- 
tions may not apply equally well to all sites, they cannot be accepted if 
there is unexplained contradictory evidence from one or more sites. 

Discussion 

The varieties and combinations of methods that can be used within 
qualitative inquiry make it a very flexible research approach, much more 
flexible than the ethnographic subset of qualitative inquiry. Ethnography, 
with the criteria established for it by members of its founding discipline, 
anthropology, requires more resources than many educational policy researchers 
and evaluators have available to them. It is particularly difficult for 
them to spend an entire school year, or more, conducting the fieldwork for 
a single study, especially if the bulk of the fieldwork is to be conducted 
by senior investigators. Furthermore, ethnographers are frequently less 
inclined to focus on the policy questions identified by research contractors 
or other audiences than to pursue questions they identify themselves in the 
field. Those questions frequently revolve around such disciplinary concerns 
as the rules, norms, and habits of a particular group or Other contextual 
factors rather than manipulable factors that affect educational program imple- 
mentation or effectiveness. 
J 

The availability of many kinds and combinations of data collection and 
analysis strategies in qualitative inquiry give researchers a wealth of 
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alternatives to select from when designing studies. Within single- or 
multiple-site studies, there are many intuitive, procedural, and interper- 
sonal strategies t^at can be combined in a variety of ways. An awareness 
of this variety allows a researcher to design a study to meet specific 
research questions, decision p-^uposes, resource constraints, and timelines 
without unduly compromising the researcher's principles. Variety also 
facilitates the use of multiple methods of inquiry. This allows the re- 
searcher to capitalize on the strengths of one method to overcome the 
weaknesses of another. And, since different methods illuminate different 
aspects and perceptions of a social setting, a more complete portrait of 
the setting can be pieced together. 

During recent years, considerable attention has been paid to the 
alternative kinds of qualitative inquiry which are available. Many metho- 
dological articles and books have been written, A great variety of quali- 
tative research and evaluation studies have been conducted. They include 
relatively pure ethnographic case studies, short-term comparative studies, 
and studies that mix qualitative and quantitative methods in diverse ways. 
Moreover, after a decade of experimenting with different approaches to 
qualitative inquiry, the research community seems to be entering a period 
of reflecting on the methods used and attempting to strengthen them. 
Several scholars have recently initiated studies to review recent quali- 
tative studies, codify the methods of data collection and analysis employed, 
and identify the ones that seem to be the most exemplaty and most often 
used. These efforts will no doubt contribute to the accumulation of 
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knowledge about qualitative inquiry. However, they are unlikely to 



resolve all of the methodological concerns that need to be addressed. 
Those concerns include: 

• How can various combinations of methods be used most effectively' 
What kinds of methods seem especially capable of compensating for 
one another? What combinations can be used without putting an 
unbearable strain on research resources? 

• How can apparently discrepant data elicited throug! different 
methodologies be resolved? What procedures can a • esearcher 
use to determine that the data produced by some methods are more 
valid, or simply represent different aspects or perceptions of 
reality? If the latter, how can a coherent portrait of reality 
be put together? 

• What new approaches to data collection and analysis will simul- 
taneously optimize the use of the researcher's subjective under- 
standing and the replicability of research findings? 

• How can researchers who have been trained in quantitative inquiry 
become proficient in qualitative approaches and vice versa? Al- 
ternatively, what management practices are available to allow 
researchers with different backgrounds to combine efforts on a 
single project? 

• How can the research design process be strengthened so there is 

a better match between problems and methods and less "contamination" 
by the researchers' preferences? 

• How can qualitative methods be used flexibly without reducing the 
rigorousness of research? What procedures can be developed to 
help researchers estimate whether and how the combinations of 
methods they are using produce sufficiently rigorous answers to 
their research questions? 
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